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DETAILED ACTION 

Response to Amendment 

1. In light of Applicant's amendment to the specification, the objection to the specification 
is withdrawn. 

2. Applicant's arguments with respect to claims 1-4 and 6-20 have been considered but are 
moot in view of the new ground(s) of rejection. 

3. Applicant's arguments, see pages 10-26, filed October 25, 2005, with respect to the 
rejection(s) of claim(s) 1, 3, 4, 7, 15, 17, and 19 under 35 U.S.C. 102(e) and claims 2, 5, 6, 9, 11- 
13, 16, and 18 under 35 U.S.C. 103(a) have been fully considered and are persuasive. Therefore, 
the rejection has been withdrawn. However, upon further consideration, a new ground(s) of 
rejection is made in view of Brown (US006631423B1). 

4. Applicant's arguments, see pages 16, 21, 22, 25, and 26, filed October 25, 2005, with 
respect to Claims 8, 10, 14, and 20 have been fully considered and are persuasive. The 35 
U.S.C. 103(a) rejections of Claims 8, 10, 14, and 20 has been withdrawn. 
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Claim Rejections - 35 USC §103 

5. The following is a quotation of 35 U.S. C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or 
described as set forth in section 102 of this title, if the differences between the subject 
matter sought to be patented and the prior art are such that the subject matter as a whole 
would have been obvious at the time the invention was made to a person having ordinary 
skill in the art to which said subject matter pertains. Patentability shall not be negatived 
by the manner in which the invention was made. 

6. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 
(1966), that are applied for establishing a background for determining obviousness under 35 
U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3 . Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating obviousness 
or nonobviousness. 



7. Claims 1-4, 6, and 7 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Schenk (US 20040003370A1) in view of Brown (US006631423B1). 



8. With regard to Claim 1, Schenk describes identifying a set of commands to be submitted 
to a processing unit {associated code that can be executed to render the art asset, [0087], render 
method data elements transferred to the GPU, [0082]); selecting a subset of the set of commands 
{pointer to a byte code stream that must be executed to render the model, [0088]); submitting the 
subset of the set of commands to the processing unit for processing {render method data 
elements transferred to the GPU, [0082]); and analyzing processing performed by the processing 
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unit in response to the subset of the set of commands {performing data transfer 
optimizations... simulating the contents of the GPU memory over the executing of the rendering 
of a model, [0115]). 

However, Schenk does not teach that the set of commands were captured and saved as 
they were previously submitted in the processing unit. However, Brown describes that the set of 
commands were captured and saved as they were previously submitted in the processing unit; 
and analyzing processing performed by the processing unit in response to the set of commands 
(the same state is again changed to the original values to render still additional primitives, a 
continuous series of primitive command sets that are to be rendered with the same state values 
appears in the captured graphics call sequence with one or more graphics state calls preceding 
them, Col. 4, lines 19-39; captured graphics call sequence are optimized by the compiler 410, 
Col. 15, lines 21-35). 

It would have been obvious to one of ordinary skill in this art at the time of invention by 
applicant to modify the device of Schenk so that the set of commands were captured and saved 
as they were previously submitted in the processing unit as suggested by Brown because Brown 
suggests that this significantly reduces the toggling of graphics states and the processing 
overhead associated with such state changes (Col. 4, lines 19-39). 

9. With regard to Claim 2, Schenk does not teach that the analyzing comprises measuring an 
amount of time taken for the subset of the set of commands to be processed. However, Brown 
describes that the analyzing comprises measuring an amount of time taken for the subset of the 
set of commands to be processed (Col. 12, lines 15-27). 
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It would have been obvious to one of ordinary skill in this art at the time of invention by 
applicant to modify the device of Schenk so that the analyzing comprises measuring an amount 
of time taken for the subset of the set of commands to be processed as suggested by Brown 
because Brown suggests that this is needed in order to optimize the performance to reduce the 
execution time (Col 12, lines 15-27). 

10. With regard to Claim 3, Schenk describes that the analyzing comprises showing how a 
scene would be drawn without texture or light, with textures and lighting, and skinned with 
textures and lighting [0120] (Figures 3a, 3b, 3c). Therefore, Schenk discloses showing how a 
scene would be drawn using only the subset of the set of commands. 

1 1 . With regard to Claim 4, Schenk describes that the processing unit comprises a graphic 
processing unit (120, Figure 6) [0046], and wherein the set of commands comprises commands 
to be submitted to the graphics processing unit to have a frame drawn (render method data 
elements transferred to the GPU, [0082], various elements of geometric data are coupled with a 
shading program at runtime to draw the asset, [0020]). 

12. With regard to Claim 6, Schenk does not teach setting the processing unit, prior to 
submitting the subset of the set of commands to the processing unit, to a particular state, wherein 
the particular state is a same state as the processing unit was in at the time capture of the set of 
commands began. However, Brown describes setting the processing unit, prior to submitting the 
subset of the set of commands to the processing unit, to a particular state, wherein the particular 
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state is a same state as the processing unit was in at the time capture of the set of commands 
began (Col. 4, lines 19-39). This would be obvious for the same reasons given in the rejection 
for Claim 1 . 

13. With regard to Claim 7, Schenk describes removing redundant transfers of code data to 
the GPU [0115]. Therefore, Schenk describes modifying one or more of the subset of the set of 
commands prior to submitting the subset of the set of commands to the processing unit. 

14. Claims 9 and 1 1-13 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Schenk (US 20040003370A1) in view of Brown (US00663 1423B1), further in view of Thelen 
(US006557167A). 

15. With regard to Claim 9, Schenk describes one or more computer readable media [0128]. 
Schenk describes removing redundant transfers of code data to the GPU [0115]. Therefore, 
Schenk inherently discloses having one or more instructions that, when executed by one or more 
processors (render method data elements transferred to the GPU, [0082]), causes the one or 
more processors to modify the stream of commands (removing redundant transfers of code data 
to the GPU, [0115]); and submit the modified stream of commands to the processing unit. 

However, Schenk does not teach that the stream of commands were captured and saved 
as they were previously submitted to a processing unit. However, Brown describes that the 
stream of commands were captured and saved as they were previously submitted to a processing 
unit (Col. 3, lines 14-20; Col. 4, lines 19-39), as discussed in the rejection for Claim 1. 
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However, Schenk and Brown do not teach determining a difference between a first 
amount of time required by the processing unit to process the stream of commands and a second 
amount of time required by the processing unit to process the modified stream of commands. 
However, Thelen describes calling and executing different sequences of code segments (Col. 6, 
lines 6-23). Therefore, Thelen describes modifying the stream of commands and submitting 
modified stream of commands to the processing unit. Thelen describes determining the amount 
of time required by the processing unit to process the different streams of commands (Col. 6, 
lines 3 1-60), and the user analyzes the results (gather and allow analysis of the performance 
data, Col. 6, lines 6-8) to determine performance bottlenecks (computer program performance 
analyzer 131 is a tool that is used to gather performance data for the computer program 125 to 
allow a user to identify performance bottlenecks and other pertinent characteristics of the 
computer program, Col. 3, lines 31-35). Therefore, the user inherently determines a difference 
between a first amount of time required by the processing unit to process the stream of 
commands and a second amount of time required by the processing unit to process the modified 
stream of commands. 

It would have been obvious to one of ordinary skill in this art at the time of invention by 
applicant to modify the devices of Schenk and Brown to include determining a difference 
between a first amount of time required by the processing unit to process the stream of 
commands and a second amount of time required by the processing unit to process the modified 
stream of commands as suggested by Thelen because Thelen suggests that in order to determine 
the performance bottlenecks that are cause by interactions between multiple code segments (Col. 
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1, lines 62-65), the difference between the first amount of time and the second amount of time 
must be determined (Col. 5, lines 1-24). 

16. With regard to Claim 11, Schenk describes that to modify the stream of commands is to 
remove one or more redundant commands (removing redundant transfers of code data to the 
GPU, [0115]). 

17. With regard to Claim 12, Schenk describes that not all variables used by a shader or GPU 
can be made available through a compiler preprocess. For example, data such as transformation 
matrices are not available at compile time. They could be communicated as implicit variables. 
This would restrict the user from extending the set of such variables. The render method 
specification is extended through external linkage. By adding the extern keyword to a variable 
declaration along with an assignment of the form=variable_name, a reference is made to an 
external variable so that the internal variable is revealed [0067]. According to the disclosure of 
this application, an internal variable is a variable within the GPU (page 26, lines 6-9). Therefore, 
Schenk describes that to modify the stream of commands is to change one or more instructions of 
an internal program of the processor to reveal a value of an internal variable of the internal 
program. 

18. With regard to Claim 13, Schenk does not teach that the instructions further cause the one 
or more processors to set the processing unit, prior to submission of the modified stream of 
commands to the processing unit, to a particular state, wherein the particular state is a same state 
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as the processing unit was in at the time capture of the stream of commands began. However, 
Brown describes that the instructions further cause the one or more processors to set the 
processing unit, prior to submission of the modified stream of commands to the processing unit, 
to a particular state, wherein the particular state is a same state as the processing unit was in at 
the time capture of the stream of commands began (Col 4, lines 19-39). This would be obvious 
for the same reasons given in the rejection for Claim 1. 

19. Claims 15, 17, and 19 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Schenk (US 20040003370A1) in view of Glennon (US005805173A). 

20. With regard to Claim 15, Schenk describes one or more computer readable media [0128] 
having one or more instructions that, when executed by one or more processors, causes the one 
or more processors to capture a plurality of commands submitted to the graphics processing unit 
in order to draw a frame of video {various elements of geometric data are coupled with a 
shading program to draw the asset, [0020], geometric data are converted into data structure, 
and a code stream is assembled that describes the manipulations required to render these data 
structures, [0021]); and save the captured plurality of commands {code is stored, [0128]). 
Schenk also describes a graphics processing unit {render method data elements transferred to the 
GPU, [0082]). 

However, Schenk does not teach capturing a state of the graphics processing unit and 
saving the captured state. However, Glennon describes capturing a state of the graphics 
processing unit and saving the captured state {Controller 108 contains control logic 260 which 
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controls the video processing. The control logic 260 also has a set of state registers. The 
control logic traverses control structure list 130 and retrieves each control structure in turn. 
The values in each control structure are loaded into the state registers, Col. 9, lines 45-53). 

It would have been obvious to one of ordinary skill in this art at the time of invention by 
applicant to modify the device of Schenk to include capturing a state of the graphics processing 
unit and saving the captured state as suggested by Glennon because Glennon suggests that a state 
of the graphics processing unit needs to be captured in order to control the formatting and routing 
of graphics data (Col. 15, lines 25-43). 

21 . With regard to Claim 17, Schenk does not teach that to capture the state of the graphics 
processing unit is to obtain the settings of all registers of the graphics processing unit. However, 
Glennon describes that to capture the state of the graphics processing unit is to obtain the settings 
of all registers of the graphics processing unit (Col. 9, lines 45-53). This would be obvious for 
the same reasons given in the rejection for Claim 15. 

22. With regard to Claim 19, Schenk describes identifying a memory location reference by 
one of the variables; and capturing the contents of the memory location {the data for every 

variable is either known, or an external symbolic reference is known that will resolve to the 

( 

memory location of that data at run time, [0101]). Drawing a model is done by setting the 
required control variables [0054]. Therefore, the variables are considered to be the same as 
commands. Therefore, to capture the plurality of commands is to identify a memory location 
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referenced by one of the plurality of commands; and capture the contents of the memory 
location. 



23. Claim 16 is rejected under 35 U.S.C. 103(a) as being unpatentable over Schenk (US 
20040003370A1) and Glennon (US005805 173 A) in view of Prindle (US 20030232648A1). 

Schenk and Glennon are relied upon for the teachings as discussed above relative to 
Claim 15. Schenk describes that the one or more instructions further cause the one or more 
processors to perform the captures and save (upload into memory, [0116]) in response to a 
request to capture the frame (CALL operations, [0107, 01 17]). 

However, Schenk and Glennon do not teach that the request is received from a remote 
computing device. However, Prindle describes that the one or more instructions further cause 
the one or more processors (GPU 96, [0104]) to perform the captures and save in response to a 
request to capture the frame {capturing video signals, [0022]), wherein the request is received 
from a remote computing device {captured video signal is sent from the CPU and passed to the 
requested device, [0034], client system 22, executable machine code 28 is read from a network 
server [0061]). 

It would have been obvious to one of ordinary skill in this art at the time of invention by 
applicant to modify the devices of Schenk and Glennon so that the request is received from a 
remote computing device as suggested by Prindle because Prindle suggests that this is needed in 
order to perform real-time videoconferencing. For example, a first video game console can 
directly link with a second video game console [0021]. 
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24. Claim 18 is rejected under 35 U.S.C. 103(a) as being unpatentable over Schenk (US 
20040003370A1) and Glennon (US005805173A) in view of Brown (US006631423B1). 

Schenk and Glennon are relied upon for the teachings as discussed above relative to 
Claim 15. 

However, Schenk and Glennon do not teach that the one or more instructions further 
cause the one or more processors to capture timing data regarding how fast portions of the frame 
of video are drawn. However, Brown describes that the one or more instructions further cause 
the one or more processors to capture timing data regarding how fast portions of the frame of 
video are drawn (Col. 12, lines 15-27; Col. 14, lines 20-24). This would be obvious for the same 
reasons given in the rejection for Claim 2. 

Allowable Subject Matter 

25. Claims 8, 10, 14, and 20 are objected to as being dependent upon a rejected base claim, 
but would be allowable if rewritten in independent form including all of the limitations of the 
base claim and any intervening claims. 

The following is a statement of reasons for the indication of allowable subject matter: 

26. The prior art taken singly or in combination do not teach a method comprising identifying 
a set of commands to be submitted to a processing unit, wherein the set of commands were 
captured and saved as they were previously submitted to the processing unit; selecting a subset 
of the set of commands; submitting the subset of the set of commands to the processing unit for 
processing; analyzing processing performed by the processing unit in response to the subset of 
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the set of commands, wherein if one or more recommendations for using the processing unit are 
violated by the set of commands, then issuing a warning identifying both the selected 
recommendation that has been violated and how much faster the frame could have been drawn if 
the selected recommendation had not been violated, as recited in Claim 8. 

The prior art also does not teach one or more computer readable media having 
instructions that, when executed by one or more processors, causes the one or more processors to 
modify a stream of commands that were captured and saved as they were previously submitted to 
a processing unit; and determine the first amount of time required by the processing unit to 
process the stream of commands and the second amount of time required by the processing unit 
to process the modified stream of commands and determine a difference between them, as 
recited in Claim 10. 

The prior art also does not teach or suggest one or more computer readable media having 
instructions that, when executed by one or more processors, causes the one or more processors to 
modify a stream of commands that were captured and saved as they were previously submitted to 
a processing unit; determine a difference between a first amount of time required to process the 
stream of commands and a second amount of time required to process the modified stream of 
commands; wherein if one or more recommendations for using the processing unit are violated 
by the stream of commands, then issue a warning identifying both the selected recommendation 
that had been violated and an indication of the difference, as recited in Claim 14. 

The prior art also does not teach one or more computer readable media having 
instructions, that, when executed by one or more processors, causes the one or more processors 
to capture a state of a graphics processing unit and a plurality of commands submitted to the 
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graphics processing unit in order to draw a frame of video; and save the captured state and 
plurality of commands, including determining whether the memory location was referenced by a 
previous one of the plurality of commands; if the memory location was not referenced by a 
previous one of the plurality of commands, then capture the contents of the memory location; 
and if the memory location was referenced by a previous one of the plurality of commands, then 
check whether the contents of the memory location are the same as the contents of the memory 
location when the memory location was referenced by the previous command, and capture the 
contents of the memory location only if the contents of the memory location are not the same as 
the contents of the memory location when the memory location was referenced by the previous 
command, as recited in Claim 20. 

27. The closest prior art (Brown US00663 1423B1) teaches that the processing unit comprises 
a graphics processing unit (304, Figure 3), the method further comprising analyzing the set of 
commands; determining, based on the analysis of the set of commands, whether one or more 
recommendations for using the graphics processing unit are violated by the set of commands; if 
one or more recommendations are violated by the set of commands, then selecting one of the 
violated recommendations (Col. 3, lines 36-48); determining how much faster the frame could 
have been drawn if the selected recommendation had not be violated (Col. 12, lines 15-27). 
However, Brown does not teach issuing a warning. 

28. Another prior art (Davidson US006446029B1) teaches that if an instruction pipeline 
stage requires more time to complete than indicated by its corresponding threshold value, then 
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thresholder (520, Figure 5B) asserts a threshold event signal 526 that is collected by an event 
counter 530 in the performance monitor (Col. 8, lines 62-66). However, Davidson does not 
teach that the warning identifies how much faster the frame could have been drawn if the 
selected recommendation had not been violate^. 

29. Another prior art (Thelen US0065 57 1 67B 1 ) teaches the user can analyze the set of 
commands by specifying queries on the performance data, and the computer program is 
"replayed" by stepping through the graphical representation, gathering information that satisfies 
the query at each step (Col. 3, line 63-Col. 4, line 5). Thelen describes determining, based on the 
analysis of the set of commands, whether one or more recommendations for using the processing 
unit are violated by the set of commands {identify performance bottlenecks by defining program 
execution conditions 132 and running computer program 125 under those defined conditions, 
Col. 3, lines 3 1-38). Thelen describes identifying performance bottlenecks by measuring the 
execution times of different sequences of code segments (Col. 5, lines 1-23) so that the user can 
inherently compare the difference between the execution times, and the user inherently uses the 
sequence of code segments that takes the shortest amount of time to execute (Col. 6, lines 31- 
60). 

Thelen describes calling and executing different sequences of code segments (Col. 6, 
lines 6-23). Therefore, Thelen describes modifying the stream of commands and submitting the 
modified stream of commands to the processing unit. Thelen describes determining the amount 
of time required by the processing unit to process the different streams of commands (Col. 6, 
lines 3 1-60). The user can modify the stream of commands and determine the amount of time 
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required by the processing unit to process the stream of commands and the modified stream of 
commands (Col 6, lines 3 1-60). Therefore, Thelen describes determining the first amount of 
time required by the processing unit to process the stream of commands and the second amount 
of time required by the processing unit to process the modified stream of commands. However, 
Thelen does not teach a stream of commands that were captured and saved as they were 
previously submitted to a processing unit. Thelen also does not teach issuing a warning. 

30. Another prior art (Krueger US006173368B1) teaches that the instructions are retrieved 
from the cache (Col. 2, lines 45-49). The instructions are accessed by referencing memory 
locations (Col. 1, lines 54-57). Indicators are set in response to new data being received by 
register 76. The new data is captured (Col. 16, lines 4-17). Krueger describes a C/D indicator 
(84) indicating whether the data is dirty or clean (Col. 14, lines 13-15). The data is dirty if it has 
been changed since it was read into register 76 (Col. 16, lines 19-26). If the data is dirty, then 
the data is captured to be written into the main memory (104, Figure 8; Col. 18, lines 47-54). 
The data is clean if the data in register 76 has been unchanged since it was read into register 76 
(Col. 16, lines 19-22). If the data is clean, then it skips step 104, and therefore skips the 
capturing of the data (Col. 18, lines 35-41). However, Krueger does not teach capturing a state 
of a graphics processing unit. 
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Prior Art of Record 



The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

1 . Davidson (US006446029B 1) teaches a method and system for monitoring the 
performance of an instruction pipeline (Col. 2, lines 35-36). 

2. Krueger (US006173368B1) teaches a class categorized storage circuit for storing non- 
cacheable data until receipt of a corresponding terminate signal (Col. 4, lines 48-62). 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Joni Hsu whose telephone number is 571-272-7785. The 
examiner can normally be reached on M-F 8am-5pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ulka Chauhan can be reached on 571-272-7782. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



Conclusion 




